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ABSTRACT 


This thesiS exaaines spectrometric oil analysis 
data in an attempt to sonstruct tables of statistical 
estinates Eielis use in evaluating a laboratory's 
performance individually and in comparison to a 
control laboratory. Tables of estimates were obtained 


from data provided by twenty six laboratories. 
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In 1956, the Naval Air Rework Facility at Pensacola 
Started a trial program to determine if spectrometric 
analysis of oil samples could be used to predict aircraft 
engine failures. The Naval Oil Analysis Program  (NOAP) 
evolved aS a consegquenss of the success of the trial 
program. The program has’ been expanded to include 
monitoring virtually all Navy lubricated systems. In 1976, 
NOAP was merged with similar Army and Air Force programs and 
became the Joint Oil Analysis Program (JOAP). References 1 
and 2 provide a more detailed background of the oil analysis 


program. 


The prediction of a pending equipment failure is 
facilitated by the spectrometric oil analysis of a sample of 
the lubricating fluid from the eguipment. The fluid sample 
is burned in the spettrometer and the concentrations of 
certain individual wearmetal elements in parts per million 
(PPM) are determined by the wavelengths of the Light 
emitted. A record maintained for each equipment contains 
the results for each wearmetal concentration from previous 
Sampies. After a sample from 2 particular piece Ose 
equipment is burned, an evaluator reviews the results of 
Current and past burns to distermine if there is an abnormal 
trend development OL abnormal concentration level. 
Depending on which wearmstals or combination of wearmetals 
have developed an abnormal trend, tne evaluator is 
frequently able to pinpoint the source and recommend the 
reguired preventive maintenance action to the equipment 
custodian. Reference 3 contains procedures, intarvals £0odc 


Sampling and normal limits of wearmetal concentrations for 





some of the equipments monitored by JOAP. 


The normal limits of wearmetal concentrations are not 
precise boundaries. One abnormal sample from a given 
equipment does not necessarily constitute a requirement for 
@ Maintenance action. Many factors must be considered by 
the evaluator. Typically a shorter sampling interval or an 
immediate resample may confirm an abnormal level or may lead 
the evaluator to conclude that some error has occurred 
(sample comtamination or sample interchange). Other factors 
are the tolerances within which a laboratory can obtain the 
Same results on repeated experiments (repeatability), and 
can Ceproduce either itsS oWn or another laboratory's results 
(reproducibility). These two factors are considered in this 


thesis. 


Intuitively, it can se seen that a laboratory must be 
capable of obtaining fairly consistent results for repeated 
burns of an oil sample. dJtherwise there would be tIlittle 
reason to expect the laboratory to provide data from which 
the evaluator would detect a discrepant equipment. We might 
expect also that the orocedure would often identify a 
properly functioning 2>Juipment as being discrepant. 
Similarly, because Military equipments are frequently 
transferred from one Location to anotaoer, it is desirable to 
be able to use the results from different laporatories for 
the same equipment without having to wait mong each 
laboratory to separately develop trend information about 
that equipment. AS a anZans of insuring that consistent 
results are provided by laboratories, a sertification 
program exists in JOAP (reference 3). Tne procedures for 
certification consider only an evaluation of a laboratory's 
individual performance. Reference 4 contains recommended 
procedures for Laboratory certification involving both an 
evaluation of a laboratory's performance and its comparison 


wich a “Control laboratocy. Table 98-2 of reference 3 





contains values of maximum allowable accuracy index and 
repeatanility index for laboratory certification. But 
questions have arisen as to whether the values in the table 


refer to repeatability or reproducibility and what is the 


Significance level of the values. Because SE these 
guestions there is some doubt about the validity of the 
numbers. 


This thesis investigates data provided by twenty six 
laboratories that utilize the Baird Atomic AE35/J-3 Atomic 
EBMLSSion Spectrometer. [he data collected are for 0, 3, 10, 
50 and 100 PPM concentrations for each of twenty elements. 
Each sample analysis is replicated fifteen times and ¢ach 
set of fifteen replications 1S Trepeated on two different 


days. 


_ 





IL. CONSIDERATIONS IN SPECTROMETRIC OTL ANALYSIS 


A. MEASUREMENT ERRORS 


As discussed in reference 5, there are numerous 
potential sources of earror in results from the atomic 
emissions spectroneter. For each observation taken (for 
each oil sample burned), the observed readiingj can be 
considered to inciude a true PPM reading plus an error tern. 
The error term can be modeled as a random variable and, for 
the moment, can be thought of as an accumulation of the 
efrects of all the possible sources of error. These sources 
include inputs such as temperature, humidity, electrode gap 
width, spectromater Standardization, inhomogeneous oil 
Samples, operator technigue, contamination, atc. The 


observed reading can be gaathematically represented by: 


Y =u +E (1) 
i i 


where Y is the result of the ith observation, 
al 


Hw is the true PPM, iad 


E is the error component of the ith observation. 
aL 


Many of the error inputs can be controlled to some 
degree py careful attention to precedures established for 
the Oil Analysis Program. These includ@ sample handling, 


spectrometer standardization and operator techniques. 
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However, in burning oil samples collected from operating 
equipments, the operator has no prior estimate of the true 


PPM of the sample nor of the error. 


From equation (1), it can be seen that if the magnitude 
of the random error is allowed to be Large relative to the 
true PPM, the error could mask a significant change in the 
estimated PPM. Since the error 1S not measurable, some 
method is needed ee, measure the performance of a 


spectrometer. 


B. MEASURES OF PERFORMANCE 


Since the random error term can be considered to be 
normally distributed (references 4, 5 and 6), good measures 
of spectrometer performance could be expected to involve 
functions of the sample mean and Sampl2 variance. To test 
whether a given spectrometer is producing results within an 
acceptable error tolerance level, one might make several 
observations on an oil sample of Known PPM concentration and 
compute the sample mean and sample variancs OF the 
spectrometer readings. If the conputed quantities were 


Within some predetermined bounds, the spectrometer could be 


considered to be operating satisfactorily. Otherwise, the 
Spectrometer Should be realigned slectronically 
(restandardized) and retested. The measures of performance 


currently used in JOAP are: 


AT = | pom ¥l (2) 
and 


2 L fe 
RI = (SUM (y. - ¥) 7 (N~1)) fe a rarer | (3) 
als ale 
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where U is the known PP4 concentration of the oil sample, 
y = SUM (y S/N), 1 = 1,...,N, is the sample nean, 
sae f 


y. is the ith observation on the sample, 
zi 


N is the total number of observations taken, 
AI is the accuracy index (true PPM - sample mean), and 


RI is the repeatability index (sampl2 standard 
deviation). 
For the reasons cited above, it was decid2i that the 
measures ot performance represented by equations (2) and (3) 


are satisfactory for use with oil analysis progran data. 


Currently usei values of the maxinum allowable AI and I 
are given in table 8-2 of reference 3. In their work with 
data obtained form the oil analsis program, D. &. Barr, T. 
Jayachandran, and H. J. Larson have found that the tabled 


values may not be realistic bounds (reference 4). 


AS current doctrine in the JOAP procedure, in operator 
makes ten sample burns from a standard o11 sample of known 
PPM concentration and uses equations (2) and (3) to compute 
AI and RI for each element under analysis. If the computed 
AI and &I do not exceed tabled values, the laboratory is 
considered to be operating within acceptable tolerances. 
Otherwise, the Operator should restandardize the 


Spectrometer and repeat the sample burns. 


i2 








A. OBJECTIVES 


The goal of this project was to statistically estimate 
the bounds or limits for AI and RI for both the within 
laboratory effect (repeatability) for a single laboratory, 
and between laboratory effect (reproducibility) for a given 
laboratory compared with a control laboratory. However, it 
Was discovered that three separate and distinct sets of 
indicies could be identified. Appendix B is devoted to a 
development of the three sets of pounds for the indices. 
Notationally, the three situations are identified as Case I, 


Case II and Case [ITI. 


Be. RESULTS 


The procedures presented in this section apply to any 
one of the twenty elements for which the Az 35/U-3 
Spectrometer is used in JOAP. They concern standardization 
With standard oil samples having any one of Q, 3, 10, 50 ofr 
100 PPM concentrations. The procedures should be applied 


for each element and conzentration of interest. 


Case I corresponis to the current usage o3f AI and 
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RI, that is, a laboratory makes a set of N sampl?2 burns with 
a standard oil sample and computes AI1 and RI1 usiag 
equations (2) and (3). The computed quantities (using n= 
10 or n = 15 Sample burns) may be compared with the bounds 
contained in table I or IV respectively of the naxt section. 
If the computed gquantiti2s are not greater than the tabled 
values, the laboratory's spectrometer may be considered to 


be operating within acceptable tolerances for Case I. 


AI1 is the magnitude of the deviation of the sample 
mean from the known PPM concentration and might be viewed as 
an indicator of spectrometer alignment or standardization. 
RI1 1S a measure of the variability in the spectrometer's 
observations. Excess variability suggests either poor 


Operator technique or an erratic spectrometer channel. 


Case II may be considered to be the tolerances 
Within which a given laboratory would be expected to 
reproduce its own results with a second complete set of N 
Sample burns from the same o11 sample. The laboratory could 
complete these procedures using any oil sample. The 
following computations would be made for the two sets of 


sample burns: 


AI2 (4) 


tl 
<I 

‘ 
aL 


and 


R12 (s /s ) (5) 
y oe 


Where ¥ is the sample mean computed from the first set of 


Sample burns, 
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X¥ 1S the sample mean computed from the second set of 


sample burns, 


Ss is RI1 computed from the first set of burns, and 
Yy 


s is RI1 computed from the second set of burns. 
x 


The computed quantities (using N = 10 or N = 15 
sample burns) from equations (4) and (5) may be compared 
With the bounds given in [Table [I or V respectively of the 
next section. If the computed value of AI2 from equation 
(4) 1S not greater than the tabled bound and if the computed 
value of RI2 from equation (5) lies between the reciprocal 
of the tabled value and the tabled value, the laboratory may 
be considered to be oparating within acceptable tolerances 


for Case Il. 


AI2 is the magnitude of the deviation of sample 
means between sets of N observations and might be viewed as 
an indicator of a change in spectrometer standization. R12 
ls a ratio of sample variances and measures a Laboratory's 


ability to reproduce its previous results. 


A word of caution concerning the interpretation of 
the index RI2 is in order. During the analyses, it was 
noted that the computed sample standard deviations from data 
sets taken from within a spectrometer appeared to be 
related. The correlation coefficent (appendix (C) was 
computed for each PPM concentration and element combination 
and tested for significance (Table VII). Because the 
correlation noted was in general quite high, it should be 
expected that in almost all applications of Case EAE 


procedures, for RI2, the laboratory will pass. 


ib) 





ce Case III 


Case III may be considered to be the tolerances 
Within which a given laboratory would be expected to 
reproduce the results obtained by a control laboratory. 
Under Case III procedures, both the control laboratory and 
the laboratory under test would complete N sample burns on 
the same oil sample. The following computation would be 
made for the two sets of sample burns. 


AI3 


II 
“<I 
) 
a 
—, 
o 


and 


RI3 (7) 


l 
am, 
7) 
> 
7) 
a) 


where yy 1S the sample mean of the control laboratory's N 


observations, 


X ils the sample mean of the test laboratory's N 


observations, 


S$ is RI1 for the control laboratory, and 


S 1s RI1 for the test laboratory. 
x 


The computed quantities (using N = 10 or N = 15 Sample 
burns) may be compared with the bounds given in table III or 
VI respectively of the next section. If the computed value 
Of AI3 from equation (6) 1S not greater than the tabled 
bound and if the computed value of RI3 from aquation (7) 
lies between the reciprocal of the tabled bound and the 
tabled pound, the laboratory under test may be considered to 


be operating within acceptable tolerances for Case III. 
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AI3 is the magnitude of the deviation between a 
control laboratory's anda given laboratory's sample means 
and might be viewed as an indicator of a given laboratory's 
alignment with respect to the control laboratory. Ri3 is a 
ratio of sample variances and measures a laboratory's 


ability to reproduce the control laboratory's results. 


C. TABLES 


Tables =I through VII are discussed in section III and in 
appendices B and =. Table VIII is a comparison of tae 
number of Laboratories for which the data submitted was used 
with the number of laboratories submitting data (see 
discussion). Table IX 15 an example of the computer output 
for one of the 100 analysis of varlance problems (100 PPM 
and element combinations). Appendices A, B and C have 
developments for aost of table IX. Tables I through VII are 
taken from the 100 comduter output pages. Tabie X is also 
taken from the computer output pages and shows for which PPM 
and element combinations the spectrometer effect was 


Significant (appendix A). 
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NO. 


2a729 
25/29 
24/25 
CCAS 
25/25 
(Ey Pas 
Papsy LFA 
CAVE 
25/29 
24/24 
22/25 
eile 
297-2 
21/28 
22/23 
hth (ars 
Fla) el a Fa 
2a) 22 
C2722 
el aaa | 


TABLE VIII 
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24/24 
fia age 
our 
2%/ 26 
20/23 
19/21 


QF LABS USED /NO. 


25 


10 
25/26 
26/26 
26/26 
CEI EL 
26/26 
26/26 
26/26 
PA fae 
26/26 
25/25 
26/25 
21/23 
26/26 
23/24 
23/24 
22/23 
22/24 
25/26 
20/23 
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UFS EAs sUeMi i’ 20 


50 
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26/26 
26/26 
21/22 
25/26 
26/26 
26/26 
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26/26 
24/ 25 
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26/ 26 
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23/24 
Zhe 3 
Zi 23 
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Lo722 
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Fag fa 
Zlf2s 
24/26 
eus2s 
14/20 
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ppm 


TABLE X 


SIGNIFICANT SPECTROMETER EFFECT 


J 
Yes 
rics 
YES 
Yas 
Yes 
Yes 
Mes 
NO 
YES 
NO 
NO 
Yes 
ves 
NO 
YES 
YES 
YES 
es 
NO 
YES 


NQ 
Y-2$ 
N9 
VES 
eS 
YES 
ECS 
wes 
NO 
NO 
NO 
NO 
NO 
NO 
YES 


NQ 


Zi, 


10 
YES 
VES 
YES 
YES 
ia) 
Mae. 
Yes 
NO 
NO 
NO 
NO 
YES 
NO 
NOQ 
TES 
Yes 
Yes 
eS 
YES 


50 
YES 
TEs 
Yes 
NO 
NO 
NQ 
Tes 
YES 
Yes 
YES 
NO 
YES 
Yes 
NO 
NO 
NO 
NO 
Yces 
Yeo 
YS 


1CC 
NO 
NOD 
YES 
NC 
NO 
NO 
NC 
GES 
YES 


NQ 
YES 











The data submitted was, in general, assuned to be a 
representative sampling from the population of AE35/U-3 


spectrometers used by JOAP. However, from X vs s plots 


(ceference 9) made from the sample means and sample standard 
deviations of the data, it appears that some laboratories 
submitted data which was not consistent with the data fron 
other laboratories. Assistance, in determining which sets 
of data appeared to be erratic or from spectrometer channels 
With set up (standardization) problems, was obtained fron 
the JOAP Technical Support Center. Tapie VIII shows tne 
number of laboratories from which the data was used compared 
with the number of laboratories submitting data. In 
general, only those sets of data for which the laboratory 


appeared to stand apart from the rest on the £ vs s plot and 
were identified by the Technical Support Center were removed 
from the analyses. Tt is felt that a few of the 
laboratories may have been more than conscientious in 
setting up the spectrometer for the sample burns while a few 
others may have been somawhat careless. If this were indeed 
the situation, the variability (caused by the two extremes) 
could have peen responsible for th2 apparent sorrelation 
noticed (table VII) for many PPM concentrations and element 
combinations. This would contribute erroneously to the sums 
of squares due to error, making the estimates larger in 


Magnitude than they should be. 


The tables of estimates ar? in a crude form: perhaps 


smoothing over elements and PPM concentrations could reduce 
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the coarseness of the values. To use the tables for PPM 
concentrations between those given, the user might use 
either linear interpolation or tleast squares regression. 
Linear least squares regression was tried using PPM as the 
independent variable ani the estimate as the dependent 
variable. In general, a good fit was obtained. However, 
some of the tabled values appear to be outliers and in those 
cases the results were not usable. Using Bartlett's test 
for equal variances (appendix B), it was found that the 
assumption of egual (homogeneous) variances was not valid, 
According to Netec and Wasserman (rfreference 9), unegual 
Variances can have pronounced effects on inferences about 
the variance components (appendix A) When using a random 
effects model. 


Future work in this area should include a repgat of this 
project with a qualified observer present at each laboratory 
for the data gathering to ensure that proper standardization 
procedures are followed. In this project, 1t was found that 
the day (standardization) effeet was significant for all PPM 
and element combinations. In many instances, the F 
statistic was more than ten times larger than the tabled F 


value (appendix A). 


Because the day effect was Significant in all cases, it 
is felt that the Case II procedures described in this report 
could be very useful to JOAP. Case I procedures (currently 
used in JOAP) used in conjunction with case II and case III 
procedures might ensure that laboratories could produce 
consistent results ont both interlaboratory and 


latralaboratory comparisons. 
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APPENDIX A 


MATHEMATICAL MODEL 


An Analysis of Variance Model (references 7 and 9) is a 
Statistical tool utilized by an experimenter to study the 
relationship between a dependent variable (an observation on 
a sample burn) and one or more independent variables (mean 
PPM concentration, spectrometers, standardization setting, 
error, etc). No assumptions are reguired about the nature 
of the statistical relation. The effects of the independent 
Variables are separately studied by partitioning the total 
Sum of squares (the summation of all the observations 
Squared) and the associated degrees of freedom into smaller 
sums of Squares that are specifically related to the 
independent variables. Dividing a sum of squares for an 
effect by its associated degrees of freedom gives an 
unbiased variance estimate (mean square). A ratio of two 
variance estimates, under the null hypothesis that the 
particular effect is z2ro, has an F distribution with 


degrees of freedon ve (numerator degrees of freedom) and up 
(denominator degrees of freedom). At a chosen level of 


Significance, the test statistic, F (the ratio of variance 
estimates), can be tested for significant effect by 
comparing F with the tabled value of an F distribution with 


- and We degrees of freeiom. A value of F greater than ofr 
equal Ee the tabled value (at the chosen level of 


Significance) implies that the effect is Slgnificant; 


otherwise, it may be considered to be zero. Analysis of 
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Variance models may be used for for fixed effects (levels of 
independent variables under study are the only ones of 
interest) or for random effects (levels of independent 
Variables under study ar2 a subset 9f a population that is 


of interest). 


The model used in analyses of the data is a Nested 
Random Effects Model (reference 7). A random effects model 
is considered appropriate because so allows EOE 
generalization of the results obtained from th2 randomly 
selected subSet of a population (Spectrometers) of interest 
to the entire population. Twenty five laboratories were 
chosen from the population of 127. The Laboratories were 
selected by drawing uniform random numbers between 1 and 127 
until twenty five laboratories had been selected. However, 
those laboratories based aboard ships and in foreign 
countries were exempted from selection. The tine reguired 
to mail oil samples to the exempted laboratories and for 


them to return the results was thought to be excessive. 


In the random selection of oil laboratories LOE 
participation in the data collection, the laboratory 
expected to be the control laboratory (JOAP Technical 
Support Center) was not selected. However, it was 
considered desirable to have the control laboratory included 
in the sampling. Hence, data were also collected from 
Technical Support Center, making twenty six laboratories in 
total. The addition of the non-randomly selected Laboratory 
to the sample should not significantly alter the 
generalization of the results to the entire population of 
Spectrometers. [ne nested feature of the model allowed the 
author to investigate the day (standardization) effect 


Within spectrometers. 


The model can be expressed mathematically as: 
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Y = u+ A + B + £ ’ 


ijk iL j6 i? ijk 
m= legac gl rf a) = APA er k = ere 4 
where Y is the kth observation from the ith spectrometer 
1 jk 
on day j, 


u is the mean effect, 


A is the ith laboratory effect, 
ae 


ae is the jth day effect within laboratory 1, and 
joi 


E. is the random error of the kth observation on 
alg 
instrument i and day j. 


The assumptions af the model are that ee is 
13 
2 2 
distributed NI(0,9 ), A. is distributed Ose | and De 
2 2 
is distributed es. for each i. The notation NI(0,0 } 


means normally and independently distributed with aean O and 


2 
Variance oO , 


The assumption of a normal distribution for observations 
On oil sample burns sesms reasonable and is documented by 
previous work with oil analysis data (refarences 4, 5 and 
6). The assumption of independence is Somewhat questionable 
based on previous work. It has been found that there exists 
a dependence between certain elements and a dependence 
between PPM concentrations (references 4, 5 and one 


However, in view of the fact that all of the analyses in 
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this report were performed for one PPM concentration and one 
element at a time, the assumption of independence does not 


seem unreasonable. 


The sums of squares for the model (reference 7/7) are: 


SSM = IJK (Y ) r 
2 2 
SSA = JK*SUM (Y -Y  ) = SUM (Y ) /JIK - SSM, 
a deters eas a Les 
2 

SS(BjA) = K*SUM (Y.. - YZ, ) 

l,j 1j.e Lee 

2 2 


= SUM (¥.. ) /K - SUM (¥, ) /JK , 
a BE BE aE 


F s 


SSE = SUM. (ieee. 7 1. . 4 
geile. jue DEBE 

2 2 

pec Une ete = SON {Yo ) 7h 
1,j,/k ijk tes eee ea 
2 

Ss0Oe- SUM =. ¥.. 

Le ak ijk ‘ 
oles eee gic aay Wig 0K = lg ..e 5 {a} 


where SSM is the sum of squares due to the mean effect, 


SSA is the sum of squares due to the laboratory effect, 


SS(B|A) is the sum of squares due to the day within 


spectrometer effect, 


SSE 1s the sua of squares due to the random error 
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effect, 


SSTO is the total sum of squares, 


4 = SoM Y /t3K, YY. =S 


Po a UM YY. sdk, 
<5 Bh) bs =E QL ieee j,k ijk 


Y = SUM Y.. /K , i, j and k are defined in equation (al), 
Lj. K ijk 


I is the number of laboratories, J is the number of days 
sampled, and K is the naumber of 


laboratory and day combination. 


ceplications for each 


The mean sguares for the model (reference 7) are: 


MSA = SSA/(I-1) 


MS(B(A) = SS(BIA)/I(J-1) , and 


Note = “Sst/7id(i>1) (a 2) 


where MSA is the mean Squares due to the spectrometer 


effect, 


MS(B[A) ais the m2an Squares due lo) the day 


(standardization) within spectrometer effect, and 
MSE is the mean squares due to random error. 
The two test statistics accociated with the model are: 


ee = MSA/MS (BI A) with (I-11) and I(J-1) degrees of 


freedom and 


- = MS(BJA)/MSE with I(J-1) and IJ(K-1) degrees of 
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freedon, (a3) 
where F is the test statistic used to test for a 
A 


Significant spectrometer effect and 


no is the test statistic used to test fora significant 


day effect. 


The usual estimators for the components of variance are: 


o = MSE, 
pers 

= (4S (BIA) ~ MSE)/R, and 

“2 

g, = (ASA ~ MS (BIA))/IK. (a4) 


3's. 





APPENDIX B 


ESTIMATORS FOR AIT AND RI 


While developing bounds on AI for the three cases, it 
was found that for variance estimates the usual chi-square 


statistic was not appropriate. The sum of k independent 


2 2 


chi-square distributed random variates (nS /o) with a 
ghee a 


degrees of freedom (reference 8) 1S a chi-square randon 


rd e 


variate (nS fo ) with a= a + a, © sore. * a degrees of 


freedom. However, the variance estimates used on 


determining the bounds for AI were linear combinations 97f 
variances for which Cochran's theorem 1S not applicable. 
Using the procedures described in chapter 17 of reference 8, 


an approximate chi-square statistic can be formed as 


follows: 
u = ng/G (51) 
2 one 
where n = (SUM g.x.) /SIM (g.x%./n.), 1 = 1,...,K, 18 the 
ed 2p hae ee a 2 


approximate degrees of freedom, 
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g = g asa + g_ MS (BIA) My g NSE, 


x = MSA, Ko = MS (BIA), XS = MSE and the g 's will be 
ih 
defined in the development that follows for th2 variance 


estimates. 


A. CASE I 


A 95 percent confidence bound on AI1 (ju- YI), is 
oy 


1/2 
given by t (.975)g ’ 
n 


where Y §$is the sample nean of K observations with the ith 
ie) 


spectrometer on day j, 


g 1S 42 Variance estimate of ({u-~ ¥  ) 
ls 


NM is the approximate degrees of freedom, and 
t (.975) is the tabled value orf Student's + distribution 
n 


having n degrees of freedon. 


The variance of (u- Y. .) can be found as follows: 
13 


Gt eae Vif) = ¥ (SUM Y 9 2/K) 
ap als k 1jk 


(1/K )V(SUM u+ A + B + E 
{ od ne jS vP Aspe 


Bye! 








= 2é2 rr 
1/K (K o + Ko + SUM V(E_ ) 
A B kK ijk 


2 2 2 
=e tmOenrt © / liae6 K==) 1,.~ <> Kh 
A B 


An estimate, gj, of V( up - Y.. ) is (from equation (a4)): 
i 


g= g 4SA + g 4S (Bt A) J g NSE, 


9, = dK, 9, = (J-1)/3K and g. = (K-N)/KN, (b2) 


where I 1s the number of spectrometers used in the analysis, 
J is the number of days (2), 
K is the number of repeated observations, and 


N is the number of sample burns for which tables were 
constructed (10 orc 15). 


The approximate degrees of freedom n associated with the 


Variance estimate g is: 


n = [g, MSA + 3 MS(BIA) + g MSE]/ 


2 2 2 2 2 2 
Cg, (asa) VA {as |) Gg, (45 (BTA)) 7 ed alee + sia Quiet Zio (k= 1 | 


(D3) 


The usual 95 percent upper confidence bound on RI1 would 


2 Life 
be [n(MSE)/ x (.05) j , where n is the degrees of freedom 
n 
2 


associated with the variance estimate, MSE, and x (.05) is 
fi 
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the tabled value of a chi-sguare distribution having on 


degrees of freedom. During analysis it was found that this 
upper bound on RI1 was not realistic when compared with the 
Sample standard deviations computed from th2 data. A 
Bartlett test for equal variances was performed (reference 
9) for several PPM and element combinations. [It was found 
that for all combinations tested, the hypothesis of equal 
Variances was rejected. Therefore, another method for 


estimating the upper bound on RI1 was developed. 


To estimate the upper bound on RI1, the sampl2 standard 
deviations for oboth days on all spectrometers were used to 
construct an empirical cumulative distribution function. 
The 95 percent upper bound was found by linear interpolation 


on the empirical cumulative distribution function. 


To find the upper bound on RI1 for N = 10 sample burns, 
1t 1s assumed that the upper tail of the distribution of 
Sample variances can be approximated by the upper tail of a 
nocmal distribution. To develop a relation between the 
distributions for 10 and 15 sample burns, it is further 
assumed that the sample variances have first and second 
moments that are related in the same manner as those of 


chi-square variates. The variance of a chi-square variate 


2 2 
(VS fo) is 2v. (reference 7), and 
sie o a 


2 ¢ 2 2 2 4 
Vis = 0 /f/V¥ V{(V f Oo = O /V 
( a! / 1 ( 1 2 / 1 


and, from the 95 percent upper bound we have 


P(S SRI) = .95 
1 1 


4 L2 2 2 4 1/2 
or, P((S.- 9 )/(2 o /v ) S$ (RI- o)/(2 0 fv) ) = .95 


3g 








2 2 4 1/2 
OE » @((RI -o 7 (2. 6 (ie) ) = .95 


2 
where S| is the estimate of sample variance for N = 15 


sample burns, 


G is the degrees of freedom of the estimate (v = 14), 


and 


@ is the standard norcmnal CDF. 
Atter equating the above to a Similar statement for N = 10 


Sample burns, we have: 
t fe = 1/2 2 1/2 
Sasa = ((14/39) eee 6 2) ) o ) 


2 
Replacing o by its estimate (MSE) gives the upp2r bound on 


RI1 for N = 10 sample burns. 


B. CASE If 


A 95 percent confidence bound on AI2 (jy - xX!) 1s given 


1/2 
my = (.975)g , where y is the sample mean of the first 
n 


set of N observations, and x is the sample mean of the 


second set of N observations. 


The variance of (Y o- Y  ####=) scan be found as follows: 
5s ie oe 
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WV ro) ) = V(Yeee) + V(¥ 8) = 2Cov(Y  ,Y ) 
Lj. 2 ly es Sag ie ny (Lae ise (bj 
2 
Commer uso) tea(tas ~W){Y., =u) = Oo , 
ae alien aeisay i: va wos alg ae A 
2 2 
Hence, ¥V{Y ~-fF .)=F=20 +20 15. 
ea) ce La B 
An eStimate, g, of V(Y - une ) is (from equation (a4)): 
<li ng |S 
g = 9 MSA + g MS(BIA) + g MSE 
q. = Q, ches = 2/K, and ae = 2 (K-N) /KN (b4) 


The approximate degrees of freedom, n, are found using 


equation (D3) with the g's as defined in equation (b4). 
z 


2 2 
A 95 percent confidence interval on RI2 (Ss /s ) is given 
Ve oe 


by: 


1/F (oo) Sees eRiZ =< Ff {.9 75) (d5) 
ni,ne ni,né 


where F ; 62979) is th2 tabled value of an F distribution 
nt,n 


having nt and n@ degrees of freedoa. 


Since the number of observations, N, is the same for 
both sets of observations nt = n@ = N-1 = 9 or 14. 
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A 95 percent bound on AI3 (ly - XI) =j%is given by 


1i/f2 
i (2.979) 9 ; 
ay 


where Yy 1s the sampl2 mean of the control laboratory's N 


observations, and 


xX is the sample mean of the given laboratory's N 


observations. 


The variance of (YY. -f. +) is £ound as follows: 
nls ey es 
WAY > ee 
33} al les ae 
=V(Y ) + V(Y. ey 2 Cor (in a 
ale pe Ge ede io. 
Cov (Y. PNA ) = 0 


2 2 2 
(eC i ee ) = 2 og. 2 og tt 2G / 15 
ag ale a) as A B 
An eStimate, g, of V(Y o- ¥. + #+~+?) is (ftom sguation (a4)): 
i. play [ee 


= BE 
g g SA + g, MS (BIA) + g NSE, 
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Gg, = 2/dK, 9, = 2(d-1)/5K, and g_ = 2(K-N)/KN (b6) 


The appropriate degrees of freedom, n, are found using 
equation (D3) with the g's as defined in equation (b6). 
I: 


The 95 percent bound on RI3 is found using equation (b5). 
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APPENDIX C 


CORRELATION 


During analysis yg the data provided by the 
participating laboratories, the sample standari deviations 
were computed and the correlation coefficient (reference 3) 


Was computed as follows: 


Chum = SUM x y 
1 


- SUM x SUM y/N 
BE 5 ee & L 


* 


At 


2 2 2 2 i/e 
rdenom = ((SUM.x ~(SUM.x.) /N) (SUM y.-(SUM yy.) /N)) 
ad ee signe 1 ee 


f= LERuM/Eaenom ». L-= loe.<>) 1 (c1) 


where y is the sample standard deviation computed from the 
1 


data collected on the first day that data was collected for 


each PPM and element combination, 


x 1s the sample standard deviation computed from the 
as 


data collected on the second day, and 


I is the number of Laboratories submitting data for a 


given PPM concentration. 
Table VII gives the values of the correlation 


coefficient xr for each PPM and element combination. To 


determine whether the correlation coefficents computed were 


uy Uy 








Significant, the test statistic t was computed as follows: 


12 2 1/72 
= E¢L = 2) 7 Ee) (5) 


Under the hypothesis that the correlaticn is zero, t is 
distributed as Student's t distribution with I - 2 degrees 
of freedom (reference 9). Entering a t distribution table 
With the test statistic t and I- 2 degrees of freedon 
yielded the tail areas that are also given in Table VII. ff 
one chose .05 as the desired level of significance, any tail 
area listed in Table VII not greater than .05 would imply 
that the corresponding correlation coefficient is 
Significant: there is a Significant poSitive correlation 
between the standard deviations of the first and _ second 
day's observations at a given laboratory. The discussion 
above assumes that the day one and day two sample standard 
deviations are observations on a bivariate normal population 
(ceference 9). However, sample standard deviations are not 
normally distributed. Hence, the results in table VII 


Should only be interpreted as rough indicators. 
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APPENDIX D 


PARTICIPATING LABORATORIES 


the following is a list of the laboratories that 
cooperated by making the the sample burns and submitting 
their results for the author's analyses. Th2 numbering 


system has no relation to the oil analyses progran. 


Lab no. Location of laboratory 
1 Technical Support Center, Pensacola 
2 McClellan AFB 
3 Hunter AAF 
4 NAS Norfolk 
5 MCAS Cherry Point 
6 Langley AFB 
7 Pease AFB 
8 Laughlin AFB 
= Bergstrom AFB 

10 NAS Whidbey Island 
11 Dover AFB 

liz FT Rucker 

13 Dyess AFB 

14 Myrtle Beach AFB 
3 ANG, Jacksonville, Fl 
16 Davis Monthan AFB 
17 McChord AFB 

18 Shaw AFB 

19 Eglin AFB 

20 Holloman AFB 

21 Randolph AFB 

22 Minot AFB 
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23 
24 
25 
26 


NAS Alameda 
Ft Hood 
Grandfork AFB 
Ellsworth AFB 
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